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ABSTRACT 

Ninety-eight medical students rated themselves and 
their fellow students using the Physician Performance Rating Scale 
(PPRS) • Two factors were studied: a knowledge factor (judgement, 
ability) and a relationship factor (appearance, interpersonal 
relationships, ability to communicate) • Reliability between peer and 
self ratings was significant but low* Students rated themselves 
generally lower than did their peers, especially in interpersonal 
skills* Peer ratings were highly significantly related to grade 
actually attained* The relationship factor was inversely related to 
grade outcome* (SH) 
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The. ulttmato criterion of: n physician's perfornmnca is tho quality of care which 
hci delivers , Much has beo.w written ahout mcuiMurenient of these t\;o variables. Even 
if qvality of cnrc cnn be OBtlmated accurately, the intervening years between medical 
school and practice make it difficult to identify markers of success. Grades report- 
edly (1) have had little predictive validity, neither has our system of credential- 
Ing (2,3). Although student behaviors seem important, those which specifically pre- 
dict success in practice have not been clearly identified. Furthermore, there is 
still a question concerning how behaviors can best be measured and by whom. 

With the PSROs, recertif ication, and documentation of continuiid competence, the 
focus for evaluation has moved beyond the medical student to include the resident 
and the physician throughout his clinical practice. Some emphasis has been given by 
the AMA recently to self assessment in continuing education. Little attention has 
been given to this type of evaluation in medical school. Yet, It would seem that hclp> 
ing the student learn to evaluate himself would be an Important asset to him throughou 
his career. With the interest in peer review in judging quality of care. It Is also 

« 

surprising that peer evaluation has not been used more In measuring student performanc 
but where it has (A-7), it has been reported of value. 

The purpose of this study was to determine if students could identify their own 
strengths and weaknesses as viewed by their peers and as reflected in other outcome 
measures such as grades and National Board Examinations (NBE) , 

METHOD 

Ninety-eight of 102 Junior medical students participated in the study. Ratings 
were taken on a Physician Performance Rating Scale (8) containing 16 Items for peer 
and 13 items for self evaluation. Items were scored on A-polnt jicales from below 
average to outstanding, with higher scores being more favorable. The scale yields two 
factors 1 A Knowledge Factor of 6 items (fund of knowledge, clinical judgment, IntoUe 
tual Curiosity, technical ability, potential as a physician, and desirability as a 



aa a cousultant), nnd a Relationship Factor with 10 items (conscientiousness, loader- 
ship, professional appcarnnco, doctor-patient rolationship , team spirit, interpersonal 
rolatlonshlprt, integrity, ability to communlcatci ideasi, desirability as a partner, and 
dosirabillty at one's own physician). The Knowledge. Factor has a test-retest value 
of r«.907 for peers and ,680 for self. The Relationship factor has an r=.902 for peerf 
and .824 for self. 

Students rc.t.jr d through their surgical clerkship in small groups. Each student 
rated himself nnd .is peers at the end of the 12 week rotation. This provided 928 
ratings. Ratings from all peers were averaged for each student to compare with his 
own evalu.ition of himself on the two factor scores. 

Average grade on quiz and essay exams > grade given by house staff, grade by tutor 
final grade in surgery, and scores on the surgical section of NBE were used as other 
outcome variables. Ratings from peers were compared with self ratings by Pearson 
cor>.«>lation. Peer and self ratings were used to predict all other grade type outcomes 
singly and in coiiibination, by multivariate analysis of covariance, and all measures 
were factor analysed to determine the dimensions In evaluation. 

RESULTS 

Reliability between peer and self ratings was significant but low. On Knowledge, 
peer and self correlated at P<.01 (r«.408). For Relationship, the correlations was 
only r=.297 which was statistically significant at the 5% level. Self ratings tended 
to bo lower than those assigned to the person by his peers, indicating stude^nts rated 
themselves as less favorable. In addition, students and peers agreed less about their 
Interpersonal skills than th-^v did about their knowledge. 

Table 1 shows how peer and self factors predicted all grade type variables taken 
In combination. Only peer ratings were significantly rclat':d to theso vavii>blos and 
at highly significant levels. The Relationship Factor was related to grado outcome 
inversely. In other words, those judged by peers to perform best in interpersonal 
skill recGlVful plgnl f Ic-'intly lower overall evaluation. 
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'flic second part of Table 1 shows self and peer factors as they predicted scores 
on the surgical section of NliK, alone. Both peer and self Knov/ledge Factors predicted 
scores on NBH significantly. Although the RolntJonship Factors are not statistically 
significant, they are again inversely related to outcome. 

The last part of Table 1 demonstrates how these same variables related to final 
grade in surgery. Final grade was based on results from quizzes, essays, NBE, and 
ratings from faculty tutors and house staff. All factors were significant predictors 

( INSERT TABLE 1 ABOUT HERE ) 
of final grade, with peer ratings being the most predictive at a .001 level and self 
ratings at only P<.05. Again, both Relationship Factors are inversely correlated 
with final grade. 

Lastly, Table 2 gives the factor structure of the variables used in evaluation. 
Four rather strong factors emerged, suggesting independent dimensions within the varl- 

( INSERT TABLE 2 ABOUT HERE ) 
ables. Essays, NBE, house staff grades, and final grade form the first factor account 
ing for 25% of the variance, peer ratings for 21%, self ratings for 18%, and quiz and 
faculty for only 10%. 

DISCUSSION 

Firstly, students were more critical of themselves than their peers were of them. 
A case might be made for that being desirable; however, peer assessments were always 
better predictors of final grade, NBE, and the multivariate combination of all grade- 
type outcotncs. This could mean either that peer ratings were more valid or just that 
peers tended to judge students similarly to the way they were judged by faculty. 
Secondly, behavior tended to relate inversely to grades. Gcertsma and Giapman (9) 
reported similarly finding t\'/o factors (cognitive and non-cognitive) and showed stu- 
dents who scored high on one scored low on the other, llils is somewhat disturbing. 
Docs it mean that students with good behavior are less intelligent? Or, does it Imply 
that popular students are busy socializing ^nd into et-ttra-curriculsr activities and 
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thus ftpeiid lv.sB titiio. studying? 

It would seem from this study that students are able, to juduc knowledge of their 
peers, in fact, even better than faculty, lliere is no reason to believe they do not 
make equally as adequate an assessment about personality related behaviors and skills. 
The fact that interpersonal behaviors predicted grades inversely could mean that medi- 
cine has worshiped too long at the alter of academic achievement and therefore has 
passed the counds of what may be needed and necessary in a practicing physician. Most 
students entering medical school are already in the upper IQ levels, increasingly so 
each year, and the tendency toward "elitism" in scholastic achievement may not be 
related to high level performance as a clinician, but more predictiv^e of careers in 
medical research where interpersonal skills are not as vital. It would seem that the 
major emphasis should be toward defining and measuring the quality of medical care, foi 
If this end point of success is measurable, we may then be able to determine accurate 
predictors. 
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XABIK 1 

PRilDICTION OF SURGICAL CLERKSHIP OUTCOME BY PEER AND SELF FACTOR SCORES 



VARIABLES 



B WEIGHT 



F-RATIO 



P LEVEL 



GRADE TYPE VARIABLE TOGETHER 

Self Relationship -.212 

Self Knowledge .132 

Peer Relationship -.372 

Peer Knowledge 1,272 

SURGICAL SECTION OF NBE 

Self Relationship -.581 

Self Knowledge ,/t3l 

Peer Ret^itlonshlp -1,904 

Peer Knowledge 3.532 

FINAL GRADE IN SURGERY 

Self Relationsihip -.260 

Self Knowledge ,030 

Peer Relationship -.653 

Peer Knowledge 2.A95 



.497 
1.100 
4.167 
8.828 

.606 
3.961 
1.026 
15. A 89 

4.113 
6.245 
8.370 
19.978 



.776 
.378 
.003*** 
.001*** 

.439 
.051 
.315 
.001*** 

.047* 
.015* 
.001*** 
.001*** 



*P <.05 
**P <.01 
***P <.001 
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TABLE 2 

FACTOR GTRUCTURK OF AU EVALUATION MEASUIUiS 



VMIMLKS FACTOR I FACTOR II FACTO R III ' FACTOR IV 

Quiz .698 
Essay . 74/, 

NBE .670 
Resident Eval. .542 

Faculty Eval. -.66 A 

Final Grade .932 

Self Relationship .879 

Self Knowledge .896 

Peer Relationship .913 

Peer Knowledge ,855 



